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1.  Introduction 

Chemical,  Biological,  Radiological-Nuclear  and  Explosives  (CBRNE)  pose  a  real  threat 
to  Canadians  and  indeed  to  North  Americans.  The  events  of  September  1 1th,  2001  prove 
that  we  are  not  immune  to  terrorist  attacks,  and  that  our  lives  and  psychological  well¬ 
being  can  be  destroyed  in  an  instant  when  one  of  these  attacks  is  executed  on  Canadian 
and/or  North  American  soil.  The  Chemical,  Biological,  Radiological-Nuclear  and 
Explosives  Research  and  Technology  Initiative  (CRTI)  has  been  conceived  to  make 
Canada  much  more  prepared  for  these  types  of  attacks  so  that  prevention  as  well  as 
response  measures  can  be  taken  to  reduce  the  damage  that  will  inevitably  be  sustained. 
This  initiative  calls  for  the  development  of  realistic  training  to  prepare  first  responders  for 
such  situations  so  that  they  can  respond  promptly,  correctly  and  effectively. 

3DInternet  in  collaboration  with  its  various  partners,  have  developed  a  highly  realistic 
synthetic  environment  (SE),  using  leading  edge  game  development  techniques.  This 
multi-user  response  training  tool  is  both  challenging  and  engaging  to  the  end  users  which 
consist  of  first  responders,  emergency  managers  and  incident  commanders.  Using 
Geographic  Information  Systems  (GIS)  data,  a  quadrant  of  downtown  Calgary  has  been 
modeled  in  detail  so  that  users  can  navigate  this  section  during  a  hazardous  train 
derailment  scenario  with  secondary  van  explosion.  Using  the  plume  evolution  data 
provided  by  one  of  our  project  partners  Dr.  Fue-Sang  Lien  (Waterloo  CFD  Consulting 
Inc.  or  WATCFD)  and  obtained  using  physics-based  models  for  plume  dispersion  in  a 
built-up  environment,  virtual  hazardous  agents  within  the  SE  evolve  realistically  through 
the  buildings  making  up  the  Calgary  city  core.  The  realistic  time  and  spatial  development 
of  the  hazardous  plume  in  the  complex  urban  environment  allow  first  responders  and 
emergency  managers  to  be  trained  within  the  SE  to  correctly  respond  to  and  manage 
these  types  of  hazards. 

The  following  sections  within  this  report  outline  the  work  that  has  been  performed  so  far 
in  regards  to  the  SE,  and  all  the  functionality  that  has  been  developed  to  provide  a 
prototype  of  a  games-based  synthetic  environment  for  CBRNE  training  of  first 
responders  and  incident  commanders. 
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2.  Work  Completed 


2.1.  Plume  Modeling 


The  gigabytes  upon  gigabytes  of  plume  data  provided  to  3DInternet  for  use  within  the 
virtual  reality  CBRNE  training  environment  has  been  used  in  two  ways.  The  first  is  to 
provide  a  real-time  “visualization”  to  the  user.  The  user  can  click  on  the  icon  for  “plume 
visualization”  to  see  the  current  state  of  “all”  plumes  currently  in-effect  within  a 

simulated  CBRNE  incident.  Each  plume 
provides  4  visualizations  of  the  data  in  a 
“quad-view”.  These  are:  east,  south, 
perspective  and  overhead  (see  Figure  1). 
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Figure  1  -  The  perspective  plume  visualization  of  the 
train  derailment. 


The  most  important  thing  when  trying  to 
visualize  the  motion  of  real  world  plume 
data  is  preserving  the  message  that  the 
data  represents.  In  other  words,  various 
factors  in  displaying  the  data  can 
themselves  inadvertently  affect  the  way 
the  data  is  shown  to  the  audience.  Or, 
other  factors  can  actually  detract  from 
clearly  seeing  what  the  data  shows.  A 
number  of  these  factors  have  been 
identified  and  addressed  in  our  plume 
data  rendering  to  make  the  visualization 
derived  from  it  as  complete  and  as 
informative  as  possible. 


The  plume  data  is  also  “sampled”  during  the  simulation  of  the  CBRNE  incident  to 
determine  if  people  (pedestrians  and  first  responders)  are  within  a  hazard  area.  In  order 
for  the  data  to  be  sampled  in  real-time  it  had  to  be  heavily  optimized  to  fit  within  the 
primary  memory  of  the  computer,  in  such  a  way  so  that  it  could  be  sampled  very  quickly 
and  often. 


The  optimization  process  for  the  plume  visualization  included: 

•  Down  sampling  the  combination  of  plume  and  puff  data  for  high  accuracy 
measurements  close  to  the  ground  surface,  and  lower  accuracy  on  top  of  buildings. 

•  Compressing  the  down  sampled  results. 

•  Building  a  content  table  for  reducing  the  delay  time  in  accessing  any  specific  time  slice. 

•  Using  minimum  computational  resources  within  the  SE  to  read  and  decompress  the 
requested  data. 
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•  Predicting  the  toxicity  level  of  undefined  positions  such  as  underground  road  ways  and 
places  outside  of  the  plume  data  area,  by  using  existing  plume  data  information  in 
conjunction  with  appropriate  interpolation  and  extrapolation  schemes. 


2.2.  Rendering  Downtown  Calgary 

With  so  large  an  area  to  be  visible  during  the  simulation  of  a  CBRNE  incident  in  an 
urban  environment,  optimization  is  imperative  in  ensuring  smooth  frame  rate  during 
operation  of  the  SE  (Figure  2).  One  of  the  key  ways  of  reducing  the  size  of  the  project 
was  the  reusing  of  textures  across  the  buildings  in  the  scene.  For  instance,  using  the  same 
brick  texture  across  multiple  buildings 
allowed  the  saving  of  texture  memory. 

Another  way  of  optimizing  the 
visualization  of  the  urban  environment 
was  to  create  LOD  (Level  of  Detail) 
models.  The  LOD  models  consist  of  a 
series  of  models  of  the  urban 
environment  at  various  resolutions  and 
concomitant  detail  of  representation  of 
urban  structures  with  the  result  that 
lower  or  higher  resolution 
representations  of  the  urban  structures 
can  be  swapped  in  or  out  in  accordance 

to  the  distance  from  the  perspective  of  Figure  2  -  Real-time  rendering  of  downtown  Calgary, 
the  player  viewing  the  environment.  looking  at  the  Enmax  District  Energy  Centre. 

This  optimization  method  saves  both 

polygon  and  texture  data  by  using  these  simpler  (lower  resolution)  models  when  the 
player  is  far  enough  away  from  the  model  to  not  be  able  to  tell  the  difference  between  the 
two.  The  most  important  of  all  optimizations  amidst  the  high  quality  of  image  we  have 
accomplished  is  due  to  carefully  constructed  shaders,  which  were  hand  written  and 
optimized  to  achieve  the  most  quality  with  the  least  amount  of  computations.  In  these 
ways,  among  others,  we  have  accomplished  the  visual  quality  which  has  been 
demonstrated  in  the  SE. 


2.3.  Multi-User  Capability 

Multi-user  capability  was  built  into  the  SE  right  from  the  initial  design  of  the  product.  An 
infrastructure  has  been  created  that  will  allow  a  PC  to  act  as  a  “host”  for  a  multi-user 
session  and  for  other  PC’s  to  connect  to  the  host  as  “clients”.  There  is  virtually  no  limit  to 
the  number  of  concurrent  multi-user  sessions  which  can  run  since  there  isn’t  a 
“centralized”  server  storing  the  position,  orientation,  and  state  of  objects  within  the  SE. 
Instead,  this  information  is  “scattered”  across  multiple  hosts.  Each  PC  running  the  SE 
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(both  hosts  and  clients)  provide  the  IP  address  or  domain  name  of  a  “Master  Server”.  The 
Master  Server  maintains  a  list  of  all  created  “sessions”  and  the  hosts  of  all  those  sessions. 
When  “creating”  a  session,  both  the  name  and  password  for  the  session  are  required  to 
create  the  session,  and  these  in  turn  are  sent  over  to  the  Master  Server.  When  “joining”  a 
session,  the  Master  Server  is  polled  for  all  running  sessions  and  a  list  is  provided  to  the 
user.  The  user  can  select  the  session  to  join  and  will  be  required  to  provide  a  password  for 
that  session. 

As  an  example:  Training  Facility  Y  is  using  the  FRITSE  to  prepare  their  first  responders. 
A  large  room  has  been  setup  with  16  computers.  The  “supervisor”  has  his  own  PC  and  is 
running  the  Master  Server.  Four  sessions  are  currently  running,  with  4  first  responders 
working  together  within  each  of  the  4  sessions  (16  first  responders  in  total).  All  16 
machines  have  designated  the  supervisor’s  machine  as  the  Master  Server.  The  supervisor 
is  looking  at  the  Master  Server  graphical  user  interface  and  can  clearly  see  that  there  are  4 
sessions  running.  He  can  also  look  into  each  of  the  individual  sessions  and  see  which 
users  comprise  those  sessions. 

The  Master  Server  has  been  enhanced  to  include  a  “Real-Time  Supervisory  Control”  or 
RTSC  (see  Figure  3).  Currently,  the  RTSC  allows  the  supervisor  to  see  a  top-down  view 
of  the  Calgary  downtown  area  using  GIS  Mapping  Technology.  Each  running  session  has 
its  own  tab  with  embedded  GIS  Information.  The  supervisor  can  see  all  pedestrians, 

vehicles,  first  responders  and  first 
responder  vehicles  in  the  GIS  view. 
Changes  within  the  SE  of  a  session, 
such  as  the  movement  of  objects 
(people  and  equipment),  updates  the 
corresponding  GIS  view  within  the 
RTSC  of  the  Master  Server. 


The  RTSC  can  potentially  provide  a 
large  amount  of  control  to  the 
supervisor  in  triggering  events  within 
the  Simulation.  These  include  but  are 
not  limited  to: 

Figure  3  -  Screen  capture  of  the  Real-Time  Supervisory 
Control. 

•  Causing  people  to  collapse  out  of  shock,  or  causing  people  to  react  in  stress  induced 
ways. 

•  Creating  vehicular  accidents  in  response  to  chaos. 

•  Causing  other  explosions  -  e.g.,  a  fertilizer  bomb  outside  the  downtown  municipal 
building. 

•  Changing  the  wind  direction. 

•  Modifying  available  resources  such  as  first  responders  and  rigs. 
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2.4.  Controlling  and  Manipulating  Objects 


Early  on  it  was  decided  that  all  object  interaction  would  occur  using  the  “Actions  Menu” 
(see  Figure  4).  A  user  can  “look”  at  an  object  and  then  options  for  how  to  control  that 

object  or  communicate  with  it  will 
appear  within  the  Actions  Menu.  For 
example,  looking  at  a  pedestrian  will 
provide  options  for  communicating  with 
that  pedestrian.  A  pedestrian  can  be 
asked  if  they  are  ill  or  can  be  instructed 
to  follow  the  user  to  decontamination, 
booking  at  a  fire  hydrant,  the  user  can 
turn  the  stem  on  top  of  the  hydrant  or 
remove  the  steamer  cap  /  discharge  caps 
(assuming  the  user  is  holding  a  nozzle 
wrench  in  his  hand),  booking  at  a 
trauma  bag  or  hydrant  bag  allows  the 
user  the  option  to  pick  these  up  and 
walk  to  another  area  of  interest,  or  to 
open  them  up  to  see  what  is  inside. 


Figure  4  -  Looking  at  the  trauma  bag.  Actions  menu  shows 
control  options. 


2.5.  Communication 


Users  have  the  ability  to  communicate 
with  other  first  responders  (emergency 
medical  services  (EMS),  police,  and 
firefighters)  as  well  as  pedestrians. 

Again,  all  communication  actions  take 
place  within  the  Actions  Menu  as  shown  in  Figure  6  -  On-screen  VoIP  GUI. 
Figure  5.  The  user  can  look  at  a  first 

responder  or  pedestrian  and  then  options  for  what  words  should 
be  communicated  appear  within  the  Actions  Menu.  A  variety  of 
dialogs  can  be  communicated;  it  is  a  simple  matter  of  selecting 
the  appropriate  one  from  the  list  and  then  filling  in  missing 
pieces  using  drop-down  lists.  Once  the  desired  dialog  has  been 
selected,  it  can  be  spoken  using  the  text-to-speech  engine 
available  in  Windows.  Users  can  also  communicate  with  other 
first  responders  using  the  portable  radio  which  they  carry.  In 
this  case  they  do  not  have  to  be  “looking”  at  another  first 
responder.  They  simply  click  the  “Speak  into  Portable  Radio” 
button  in  the  Actions  Menu. 


Figure  5  -  Actions  menu 
showing  communication 
options. 


Page  8  of  19 
W7702-125298/001/SV 


3Dlnternet  Inc. 

First  Responder  Immersive  Training  Environment  (FRITSE):  Development  of  a 
Games-Based  Simulator  for  CBRN  First  Responder  Training. 

In  a  multi-user  session,  “real-users”  in  addition  to  using  the  dialog  options  within  the 
Actions  Menu  (for  face-to-face  as  well  as  portable  radio  communication)  are  able  to  use 
the  built-in  voice  over  IP  (see  Figure  6).  Only  one  person  is  able  to  use  the  voice  over  IP 
channel  at  a  time.  Holding  down  the  ‘P’  key  opens  up  the  channel  for  communication, 
and  the  user  is  able  to  start  speaking  through  their  microphone.  The  “other  users”  within 
the  multi-user  session  can  hear  the  speaker  through  their  headsets.  The  heads-up-display 
within  the  SE  clearly  shows  who  is  using  the  voice  over  IP  channel. 


During  the  course  of  the  simulation  of  a 
hazardous  release  incident,  several  rigs 
(apparatus)  are  required  to  complete  the 
stages  of  decontamination  and  attack  of  the 
train  derailment  fire.  These  include  the  Fire 
Engine,  Aerial  Ladder  Truck,  Hazmat  and 
District  Chief  Van.  Each  of  these  rigs  can 
have  anywhere  from  1-5  responders  within 
them.  If  we  factor  in  police  and  EMS,  then 
during  the  simulation  a  user  could  possibly 
encounter  25  other  first  responders.  First 
responders  which  are  not  controlled  by  a 
“real-user”  are  controlled  by  artificial 
intelligence  (AI)  instead.  Within  the 
framework  of  the  Simulation  Engine  these 
entities  are  known  as  NPCs  or  “non-player 
characters”.  Each  of  these  maintains  an 
internal  “task  list”,  and  performs  tasks  in  a  linear  fashion. 

When  playing  the  role  of  a  Captain,  a  user  can  instruct  an  AI 
controlled  first  responder  to  perform  certain  tasks  such  as: 

•  Putting  on  their  (Self-Contained  Breathing  Apparatus)  SCBA 
and  face-piece. 

•  Putting  on  their  firefighter  jacket  and  pants. 

•  Positioning  cones  to  delineate  an  area  for  conducting 
decontamination  (see  Figure  7). 

•  Spraying  high-volume  /  low  pressure  water  on 
incapacitated  persons  possibly  contaminated  with  a 
chemical  agent  (see  Figure  8). 

•  Bringing  an  incapacitated  person  into  the  decontamination 
area. 

This  has  the  direct  affect  of  adding  a  “task”  to  the  NPCs  internal  tasks  list.  A  “task”  is 
programmed  to  implement  a  specific  interface,  so  there  are  a  group  of  functions  which 


Figure  8  -  Firefighters  positioning 
cones  in  the  decontamination  area. 


2.6.  Artificial  Intelligence 


Figure  7  -  Showing  decontamination  setup  with  high 
volume  -  low  pressure  water. 
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are  “common”  to  all  tasks.  This  makes  programming  additional  tasks  and  maintaining 
existing  tasks  easier.  Using  this  framework  a  “library”  of  tasks  has  been  created  which 
can  be  performed  by  any  first  responder  (firefighter,  EMS  and  police). 


2.7.  Pedestrians  and  Vehicular  Traffic 

Pedestrians  and  vehicular  traffic  have  been  added  to  the 
3D  representation  of  downtown  Calgary.  Pedestrians  are 
critical  to  the  training  functionality  of  the  SE  as  they 
interact  with  the  plume  model  in  real-time  by  displaying 
the  symptoms  of  a  possible  exposure  to  a  hazardous 
material,  as  well  as  being  an  integral  part  of 
decontamination  and  subsequent  treatment  of  exposure 
to  the  hazardous  material.  Pedestrians  walk  along  paths 
inside  of  downtown  Calgary.  They  traverse  a  network  of 
waypoints  and  create  the  illusion  that  they  have  “places 
to  go  and  people  to  meet”.  When  navigating  towards  a 
specific  waypoint,  the  pedestrian  will  pick  a  “point”  x 
units  away  from  the  waypoint  in  a  random  (x,  y) 

Figure  9  -  Real-time  rendering  of  direction.  Think  of  this  as  a  random  point  on  the 
pedestrians.  circumference  of  a  circle  x  units  in  radius.  Algorithms  of 

this  nature  allow  multiple  pedestrians  walking  to  the 
same  point  to  take  slightly  different  paths  instead  of  walking  in  a  robotic  line.  When 
pedestrians  encounter  injured  people  (as  a  result  of  hazard  material  exposure  or  other 
causes)  along  their  route  they  may  try  to  help.  However,  if  lots  of  pedestrians  are 
currently  helping  an  injured  person,  they  may  carry  on  without  stopping. 

Vehicular  traffic  has  been  added  for  additional  realism.  They  bring  life  to  the  city  and 
also  act  as  occasional  annoyances  when  first  responder  vehicles  are  attempting  to 
navigate  the  city  and  arrive  at  staging  areas. 
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2.8.  Treating  Injured  Persons 


Figure  11  -  Real-time  rendering 
showing  a  collapsed  pedestrian  who 
has  suffered  a  heart  attack. 


The  train  derailment  explosion  causes  injuries  to  a 
number  of  people  on  the  south  and  north  side  of  the  4th 
street  underpass.  Some  of  these  injured  people  have 
suffered  minor  head  trauma  and  others  have  suffered 
major  head  trauma.  Other  pedestrians  will  naturally  be 
concerned  about  these  injured  people  if  they  encounter 
them,  and  will  stop  to  see  if  they  can  help.  If  a  first 
responder  is  on  scene  and  is  within  view  of  a  concerned 
pedestrian,  the  pedestrian  will  run  to  the  responder  and 
notify  him  /  her  that  someone  is  hurt  and  requires 
medical  intervention.  As  always,  the  user  (playing  the 

role  of  a  first  responder)  can  decide  the  desired  course 
of  action.  He  /  she  can  pick  up  a  trauma  bag  from  a  Fire 
Engine  and  then  approach  an  injured  person  lying  on 


the  ground.  The  injured  person  can  be  repositioned 
(while  their  heads  are  being  supported)  and  a  variety  of 
diagnostic  tests  can  be  performed  according  to  the  user’s 
discretion.  Pupil  dilation,  blood  pressure,  pulse  and 
respiration  can  all  be  assessed  by  the  first  responder  see 
Figure  10).  Additionally,  there  are  instances  where  a 
pedestrian  has  collapsed  due  to  cardiac  arrest  and  will 
need  to  be  resuscitated  using  an  automatic  defibrillator 
which  is  also  available  in  the  Fire  Engine  (see  Figure 
11). 


Figure  10  -  First  responder  is 
checking  an  injured  person's  pupil 
dilation. 
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2.9.  Decontamination 


Two  key  events  have  been  built  into  the 


Figure  12  -  Real-time  rendering  showing  a  first 
responder  communicating  with  a  pedestrian  that 
has  been  exposed  to  a  hazardous  agent. 


SE:  the  initial  train  derailment,  and  the  van 
explosion  possibly  releasing  an  unknown 
hazardous  agent  which  occurs  approximately 
1 5  minutes  later.  The  hazardous  agent  which  is 
released  into  the  surrounding  area  causes 
severe  symptoms  in  the  people  (pedestrians  and 
first  responders)  which  it  encounters.  People 
exposed  to  the  effects  of  the  hazardous  agent 
are  eventually  overcome  with  symptoms  of 
drooling,  discharge  from  the  eyes  /  nose,  heavy 
breathing  and  coughing.  Without  immediate 
first  responder  intervention,  these  affected 
people  may  die.  Playing  the  role  of  Captain,  a 
user  can  instruct  other  first  responders  to  setup 
their  rigs  for  decontamination  and  treatment. 


Fire  Engines,  Aerial  Ladder  Trucks  and 

Hazmat  can  be  utilized  for  the  decontamination  setup.  Next,  the  Captain  can  instruct  first 
responders  to  start  laying  out  cones  which  will  be  used  to  guide  the  possibly 
contaminated  people  through  the  rigs  so  that  high-pressure  /  low  volume  water  can  be 
sprayed  on  them  to  “clean”  them.  The  Captain  can  also  instruct  first  responders  to  start 
locating  other  people  incapacitated  within  “hot 
zones”,  and  start  guiding  them  through  the 
decontamination  process.  As  a  first  responder,  the 
user  is  able  to  communicate  with  people  suffering 
from  hazardous  material  intoxication  directly  to 
assess  their  condition  (see  Figure  12).  They  can 
have  them  follow  them  to  the  decontamination 
area,  have  them  remove  their  clothing,  and  then 
instruct  them  to  walk  though  the  pylons  towards 
the  other  side  where  they  will  be  checked. 

Dispatch  can  be  contacted  via  the  portable  radio 
and  a  request  can  be  made  for  transit  buses.  A 
maximum  of  5  can  be  deployed  to  the 
decontamination  area.  Once  checked  over  by 

Hazmat  and  then  EMS,  the  affected  person  will  be  Figure  13  -  Real-time  rendering  showing  a  person 
provided  a  yellow  blanket  and  will  be  instructed  to  possibly  exposed  to  a  hazardous  agent,  walking  to  a 
take  a  seat  in  one  of  the  transit  buses  (see  Figure  transit  bus  after  being  cbecked  by  EMS- 
13). 
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2.10.  Fire  Incident 


Playing  the  role  of  Captain,  the  user  can  choose  to  start  fighting  the  fire  that  may  be  due 

to  the  train  derailment.  Or  the  user  can  focus  on 
treating  injured  people  and  then  focus  efforts  on 
decontamination  and  possible  treatment. 
Alternatively,  the  user  can  call  in  a  second  alarm 
and  have  one  team  focus  on  decontamination  and 
the  other  team  focus  on  fighting  the  train 
derailment  fire.  It  is  in  the  hands  of  the  Captain, 
acting  as  Incident  Commander,  to  decide  what 
actions  need  to  take  place  and  when,  and  what  the 
priorities  are. 

First  responders  are  able  to  perform  a  variety  of 
actions  in  response  to  the  train  derailment  fire  (see 
Figure  14): 

•  Rigs  can  be  staged  at  certain  areas  to  fight  the  fire. 

•  Pony  lines  at  the  rear  of  the  Fire  Engines  can  be  dragged  over  to  fire  hydrants  to  be 
attached  to  the  steamer  caps. 

•  Bolt  cutters  can  be  used  to  cut  holes  in  fences  allowing  better  access  to  the  train 
derailment  fire. 

•  Ground  monitors  (for  blitz  fire)  can  be  positioned  for  fire  attack. 

•  65mm  attack  lines  can  be  dragged  from  fire  engines  and  connected  to  ground  monitors. 

•  Levers  at  the  rear  cabin  of  the  Fire  Engines  can  be  manipulated  to  open  /  close  water 
valves. 

•  Nozzle  wrenches  can  be  used  to  open  /  close  steamer  and  discharge  caps  on  fire 
hydrants. 

•  Flydrant  gate  valves  can  be  attached  to  fire  hydrants. 

•  The  stem  on-top  of  the  fire  hydrants  can  be  opened  /  closed  with  nozzle  wrenches. 

•  Ground  monitors  can  be  controlled  to  start  shooting  water.  Vertical  angle,  jet  /  fog 
control,  and  water  pressure  can  be  adjusted. 


Figure  14  -  Real-time  rendering  showing  the 
view  of  the  train  derailment  using  field 
glasses. 


As  a  first  responder,  ground  monitors  can  be  positioned  and 
oriented  to  shoot  water  at  the  train  derailment  fire.  Fire 
hydrants  can  be  setup  and  connected  to  nearby  Fire  Engines. 
After  65mm  attack  lines  are  dragged  from  the  Fire  Engine 
and  connected  to  the  ground  monitors,  the  first  responder  can 
then  pull  the  levers  on  the  ground  monitors  to  start  shooting 
water  at  the  fire  (see  Figure  15).  The  train  derailment  fire 
will  respond  to  each  stream  of  water  that  is  making  contact 
with  it.  After  two  streams  of  water  the  train  derailment  fire 


Figure  15  -  Real-time  rendering 
showing  a  ground  monitor  with 
65mm  attack  line  attached. 
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will  noticeably  start  to  die  down,  and  after  four  streams  of  water  the  fire  will  eventually 
be  put  out  completely. 

The  130mm  (pony  line),  65mm  attack  line 
and  44mm  attack  line  hoses  are  “dynamic” 
and  respond  to  the  movements  of  the  user 
(see  Figure  16).  Sophisticated  algorithms 
using  spline  computations  for  the 
positioning  of  the  hose,  and  skeletal 
animation  computations  to  control  the 
deformation  of  the  hoses  have  been  used. 

These  result  in  hoses  which  can  be  dragged 
by  the  user  and  maintain  a  fixed  length  as 
defined  by  the  programmer.  As  the  user 
drags  the  hose  end  farther  and  farther  away 
from  its  point  of  origin,  the  hose  will 
realistically  uncoil  and  straighten  just  like  it 
does  in  real-life. 

Figure  16  -  Real-time  rendering  showing  a  first 
responder  holding  a  44mm  attack  line  with  nozzle. 


2.11.  Reporting 


Users  can  terminate  the 
scenario  and  view  a  report  of 
all  actions  which  have  been 
performed  by  clicking  on  the 
checkered  flag  icon.  All 
actions  which  occur  in  the  SE 
by  real-players  and  AI  players 
are  sent  over  to  the  Reporting 
System  built  into  the  SE.  The 
Reporting  System  assesses  all 
actions  to  make  sure  they  are 
complete  and  have  been 
performed  at  the  correct 
moments.  As  an  example,  let 
us  imagine  that  the  user  is 
checking  the  pupil  dilation  of 
an  injured  person.  The  user  can 
Figure  17  -  An  example  of  a  report  card  after  a  training  session.  use  this  information  along 

with  other  diagnostic  tests,  to 
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determine  if  the  injured  person  has  suffered  a  major  head  trauma.  The  user  can  assess  the 
injured  person  and  submit  an  assessment  (see  Figure  17)  which  then  gets  sent  to  the 
Reporting  System.  If  the  injured  person  has  been  assessed  incorrectly  then  this  will  be 
shown  within  the  report.  Additional  grading  requirements  can  be  built  into  the  Reporting 
System  depending  upon  the  organization. 

Additionally,  items  within  the  report  card  can  be  filtered  based  on  actions  performed  only 
by  real-players  and  actions  performed  only  by  the  users  themselves.  Items  can  also  be 
filtered  so  that  only  actions  which  were  performed  incorrectly  and  result  in  point 
deductions,  are  shown  to  the  user.  All  action  items  within  the  report  are  time  stamped, 
and  more  information  about  a  particular  item  can  be  gathered  by  clicking  on  the  “More 
Info”  button.  Going  back  to  the  incorrectly  assessed  pupil  dilation  of  an  injured  person, 
the  user  can  click  the  “More  Info”  button  for  this  item  to  see  exactly  what  was  wrong 
with  the  submitted  assessment. 
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3.  Future  Work 


3.1.  Enhancements  to  the  Real-Time  Supervisory  Control 

The  Real-Time  Supervisory  Control  or  RTSC,  allows  for  an  Instructor  /  Supervisor  to 
observe  the  movement  of  all  entities  in  a  top-down  view  of  the  City.  Similar  to  a  Google 
Maps  view,  layers  built  into  the  RTSC  allow  groups  of  entities  to  be  toggled  on  /  off.  We 
propose  that  additional  functionality  be  built  into  this  tool  to  allow  Supervisors  to  create 
events  and  circumstances  within  the  Simulation  during  runtime.  This  will  offer  an 
additional  tier  of  non-determinism  within  the  Simulation.  Supervisors  will  be  able  to 
cause  van  explosions,  create  traffic  jams,  create  additional  crowds  of  people  and  create 
additional  injured  persons  on  the  fly.  While  creating  new  events  and  conditions  in  real¬ 
time,  Supervisors  will  be  able  to  assess  whether  or  not  participants  are  acting 
appropriately,  and  whether  incident  command  is  assessing  the  situation  accurately  and 
making  the  right  decisions  at  the  right  moments. 


3.2.  Improvements  to  the  Task-Based  Artificial  Intelligence 
Framework 


The  Task-Based  Artificial  Intelligence  Framework  is  used  by  NPC  (non-player 
characters)  within  the  SE.  These  AI  controlled  responders  maintain  a  list  of  "tasks"  which 
they  have  been  instructed  to  perform.  A  task  can  be  locating  the  nearest  first  responder 
rig  to  put  on  SCBA,  or  locating  an  injured  person  in  the  hot-zone  and  guiding  them  to 
decontamination.  An  entire  library  of  "tasks"  can  be  programmed  to  give  first  responder 
AI  characters  additional  behaviors  and  capabilities.  A  "Captain"  or  "Incident 
Commander"  is  able  to  delegate  tasks  to  first  responders  using  the  Task- 
Based  Artificial  Intelligence  Framework.  The  framework  itself  maintains  the  library  of 
tasks  and  those  methods  /  functions  which  "execute"  the  tasks.  The  entire  framework  is 
highly  extendable  and  has  been  designed  so  that  adding  additional  tasks  and  behaviors  is 
straightforward  and  intuitive  for  Programmers.  We  need  to  build  upon  the  framework  to 
allow  Incident  Commanders  to  be  able  to  delegate  tasks  regardless  of  whether  their 
machine  is  acting  as  a  host  (created  a  session)  or  a  client  (joined  a  session).  We  also  have 
to  implement  "task  clean-up"  so  that  if  a  first  responder  is  performing  a  task  such  as 
placing  cones  in  the  decontamination  area,  and  they  are  asked  to  perform  another  task  as 
a  priority,  that  they  perform  a  clean-up  of  the  current  task.  In  the  case  of  placing  cones,  if 
the  first  responder  is  carrying  a  cone  he  /  she  would  quickly  place  the  cone  in  the  correct 
location  or  put  it  back  before  proceeding  to  the  next  task. 
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3.3.  Building  Additional  Objectives  for  EMS  and  Police  Roles 


Currently  a  user  can  take  on  the  role  of  EMS  or  police,  however  specific  events  and 
circumstances  have  not  been  built  in  the  Simulation  to  make  these  roles  rich  and 
engaging.  Pedestrians  that  are  reporting  causalities  will  approach  fire  personnel  only,  and 
playing  the  role  of  EMS  currently  doesn't  allow  you  to  tend  to  injured  persons.  Playing 
the  role  of  a  Police  Officer  also  doesn't  offer  a  rich  user  experience.  We  need  to  program 
random  events  and  specific  objectives  for  these  roles.  These  could  include: 

•  Crowd  control. 

•  Traffic  control. 

•  Controlling  access  to  dangerous  areas  and  setting  up  barricades. 

•  Preliminary  investigation  of  the  secondary  event  that  could  lead  to  apprehending 
suspects. 

•  Dealing  with  defiant  pedestrians. 

•  Dealing  with  looters  during  the  chaos. 


3.4.  Enhancements  to  Vehicular  Traffic 

Algorithms  will  be  developed  which  create  much  more  realistic  traffic  flows  throughout 
the  City.  Vehicles  will  respond  realistically  to  other  close-by  vehicles  moving  in  parallel 
down  one  way  streets,  and  will  also  respond  to  traffic  lights.  Additionally,  vehicular 
traffic  will  be  much  more  sensitive  to  first  responder  vehicles  such  as  Ambulances,  Police 
Cars  and  Fire  Apparatus.  Drivers  of  these  vehicles  will  understand  that  they  need  to  move 
out  of  the  way  for  these  vehicles  and  provide  space.  Coupled  with  the  Real-Time 
Supervisory  Control,  Supervisors  will  have  the  ability  to  cause  vehicular  accidents  due  to 
the  prevailing  chaos  in  the  areas  where  the  train  derailment  and  secondary  van  explosions 
have  occurred. 


3.5.  Virtual  Reality  (VR)  Enhancements 

To  make  the  Simulator  much  more  immersive  and  create  those  conditions  necessary  to 
make  the  user  feel  like  they  are  actually  within  the  Virtual  Environment,  we  feel  that  the 
integration  of  the  Oculus  Rift  is  an  absolute  necessity  to  achieving  these  objectives.  Users 
will  have  available  to  them  a  stereoscopic  3D  display  with  head  tracking  and  position 
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tracking.  The  3DInternet  team  will  create  the  Software  required  to  communicate  with  the 
Oculus  Rift  so  that  real-time  rendered  images  from  the  left  and  right  eye  can  be  sent 
directly  to  the  VR  headset,  and  head  tracking  and  position  tracking  can  be  retrieved  and 
applied  to  the  transform  of  the  virtual  camera  within  the  3D  Virtual  Environment.  Once 
the  Software  is  in  place  and  functioning  with  the  rest  of  the  System,  the  user  will  be  able 
to  look  around  the  Virtual  Environment  just  like  in  the  real-world.  The  combination  of 
the  separate  left  and  right  eye  real-time  rendered  imagery  on  the  stereoscopic  display, 
will  allow  users  to  perceive  "depth"  inside  the  Virtual  Environment. 


3.6.  Advanced  Performance  Monitoring  with  Persistent 
Storage 


The  current  reporting  system  within  the  Simulation  will  receive  a  full  upgrade,  and  in 
addition  to  simply  providing  a  report  of  correct  /  incorrect  actions  within  the  Simulation, 
will  also  be  continually  monitoring  the  actions  and  reaction  times  of  participants.  A  set  of 
key  performance  indicators  will  be  identified  and  then  tracked  from  the  start  of  a  session 
right  through  to  its  conclusion.  Supervisors  will  be  able  to  leverage  the  output  of  the 
Advanced  Performance  Monitoring  System  so  that  training  can  be  targeted  to  those 
specific  areas  requiring  improvement. 

All  performance  monitoring  information  will  be  forwarded  to  a  back-end  database  for 
persistent  storage  and  later  retrieval.  Supervisors  using  a  Graphical  User  Interface  screen 
built  into  the  Real-Time  Supervisory  Control,  will  be  able  to  access  the  back-end 
database  and  lookup  any  past  training  session  to  see  exactly  what  transpired  and  the 
output  of  the  Performance  Monitoring. 

The  Advanced  Performance  Monitoring  System  (APMS)  will  provide  a  host  of  other 
metric  measuring  capabilities  to  Instructors  and  Supervisors.  The  APMS  tracks  all  actions 
and  behaviors  within  SE  sessions  and  allows  Instructors  to  see  quite  clearly 
how  participants  are  performing  by  measuring  certain  predefined  "key  performance 
indicators".  Instructors  can  then  revise  their  training  methodologies  and  provide  more 
emphasis  in  those  areas  which  appear  to  be  problematic  for  trainees.  Further  sessions 
within  the  SE  will  then  provide  additional  performance  reports  and  measurements  on  key 
performance  indicators.  The  graphing  /  charting  visualizations  within  the  AMPS  will 
clearly  show  the  increase  /  decrease  on  key  performance  indicators  as  a  function  of  time. 
If  Instructors  see  these  increasing,  then  clearly  the  sessions  within  the  SE  are  providing  a 
training  benefit. 
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4.  Conclusion 

The  First  Responder  Immersive  Training  Environment  (FRITSE)  provides  first 
responders,  incident  commanders  etc.  with  a  realistic  and  engaging  tool  to  prepare  these 
kinds  of  users  for  hazardous  scenarios.  In  particular,  users  will  experience  those 
conditions  representative  of  a  train  derailment  involving  the  release  of  a  toxic  industrial 
material  and  a  van  explosion  incident  involving  the  possible  release  of  a  hazardous 
substance  into  the  atmosphere.  Working  together  in  teams,  end  users  will  be  able  to  apply 
their  knowledge  as  first  responders  to  treat  injured  persons,  perform  decontamination  and 
fight  the  resulting  train  derailment  fire.  The  work  currently  perfonned  allows  first 
responders  to  run  the  SE  in  both  single  user  and  multi-user  modes.  The  artificial 
intelligence  built  into  the  SE  allows  for  other  non-player  first  responders  such  as 
firefighters,  police  and  EMS  to  play  their  roles  within  the  simulation  of  a  CBRNE 
incident,  and  to  also  take  instructions  from  incident  commanders  when  necessary.  This 
creates  the  sense  that  users  are  not  alone,  and  that  in  fact,  they  are  part  of  the  wider  tri¬ 
services  team  and  are  coordinating  their  actions  as  part  of  the  response  effort. 

Our  future  work  in  this  Simulation  Product  will  consist  of  improving  those  parts  which 
are  critical  in  the  functionality  and  marketability  of  the  Product.  Components  such  as  the 
Artificial  Intelligence  Engine  which  drives  the  behavior  of  NPCs  and  the  corresponding 
Task  Library  offer  significant  value  to  future  products,  as  do  the  Multi-User  Engine  and 
corresponding  Master  Server  with  Real-Time  Supervisory  Control.  The  FRITSE  provides 
a  stable  and  extensible  framework  for  additional  first  responder  training  scenarios  as  well 
as  scenarios  for  other  types  of  products  which  require  “training  within  teams”  and 
“coordinated  response”.  Efforts  will  also  be  placed  in  upgrading  the  performance 
monitoring  aspect  of  the  SE  so  that  key  perfonnance  indicators  can  be  measured  and 
areas  that  require  additional  training  can  be  identified. 
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